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A method of embedding auxiliary information 
(14) into the digital representation of publication quality 
color-component digital data (10). The method applies to 
all digital data for which individual values are represented 
by discrete numerical values, and for which a comesponding 
approximation knovm as a digital reference palette image 
(12) can be made in terms of a lesser number of discrete 
digital data values. The invention creates an intermediate, 
digital, color-component difference image (13) that allows 
steganographic methods (15) to hide or embed (15) the 
auxiliary data (14). The invention secures the auxiliary 
data (14) finom detection and from unauthorized removal or 
use by means of the digital reference palette image and a 
steganographic key. By a substantially reverse process, the 
embedded auxiliary data can be an authorized user. The 
invention provides for a means to combine a removable, 
visible digital watermark with publication quality digital image 
data. 
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REFERENCE PALETTE EMBEDDING 
FIELD OF THE INVENTION 

The present invention generally relates to digital manipulation of 
numerical data. More specifically, the invention relates to the embedding of 
5 large amounts of external data into the numerical values used to represent a 
publication quality digital image without altering the appearance of the digital 
image. This invention was made with Government support under Contract No. 
W-7405-ENG-36 awarded by the U.S. Department of Energy. The Government 
has certain rights in the invention. 

10 Many digital representations of image data have resolutions in intensity 

and color range greater than is required to represent the meaningful content of 
the information. Digital representations of pubUcation quaUty images are 
ordinarily in Truecolor format using eight or more binary bits of information, for 
each of the three primary colors (red, green, and blue), for a total of at least 24- 

15 bit resolution. An alternative publication quality format for digital images uses 
primary color complements (cyan, yellow, and magenta), and black to represent 
the image information. The pubhcation quahty of Truecolor digital images 
insures that the all the information necessary to reproduce the original image in 
print is present in the alternative electronic form. Truecolor digital images are 

20 most often the first-generation image data produced by sensors in scanners or 
electronic cameras capable of recording the highest quality images. 

In many situations, fewer than 24 bits resolve an image adequately to 
convey its meaning and content. Color reduction methods analyze a Truecolor 
image to determine a smaller number of colors that can be used to reproduce an 

25 approximation to the original publication quality image. Color reductions to 256 
or fewer colors are used commonly for digital images intended for display in 
electronic documents or via the Internet worldwide web (www). Images stored in 
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the CompuserveT^*^ Graphics Interchange Format (GIF), the MICROSOFT® 
Windows Bitmap™ (BMP), and tagged-image file format (TIFF) formats often 
use a 256-color palette. The color-reduced palette requires 8-bits per picture 
element (pixel) to approximate the original 24-bits per pixel Truecolor values. 
5 Reference palette embedding is a new steganographic method for 

manipulating the information in a 24-bits per pixel Truecolor host image, in 
order to insert auxiliary data with less error than is caused by methods that 
replace directly some of the 24-bits with the axixihary data. Reference palette 
embedding as taught here provides invisibility of the auxiliary information, in 
10 comparison with the method disclosed in U.S. Patent number 5,686,782 issued 
August 19, 1996 for DATA EMBEDDING, which is included herein by reference 
for all purposes. 

The reference palette embedding invention guarantees that the avtxiliary 
information placed into the image affects only the parts of the Truecolor image 

1 5 that are redundant, and therefore unnecessary for representing the image 

content. Methods that manipulate the picture element (pixel) values directly by 
either the methods taught in the aforementioned DATA EMBEDDING patent, or 

by the methods taught in U.S. Patent Application Serial No. 08/646,837 
filed May 8, 1996, for MODULAR ERROR EMBEDDING, also included herein 

20 by reference for all purposes, modify significantly the bit values within the image 
pixel. Hereinafter, the teachings of the above-described U.S. Patent and the 
above-described U.S. Patent Application will be referred to as DATA 
EMBEDDING process and MODULAR ERROR EMBEDDING process, 
respectively. These alternative steganographic methods necessarily affect the 

25 image content to some degree. The present invention, reference palette 

embedding, utilizes a color-reduced version of the Truecolor image as a template 
to ensure that the embedding process affects the image quahty as Httle as is 
possible. 
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Reference palette embedding uses and extends the DATA EMBEDDING 
process as taught in the above-mentioned US patent. As disclosed in the DATA 
EMBEDDING patent, the auxihary data are embedded in a manner that 
manipulates the noise component of the host data, and that does not modify 
5 directly any host data values. In reference palette embedding, as taught herein, 
the auxihary data are embedded into the difference between the original 
Truecolor image, and a color-reduced version of the original image. 

The color-reduced image and the digital key taught in the DATA 
EMBEDDING patent combine to permit the construction of the auxihary data 

10 from the modified Truecolor image. 

Data embedded into the host image with the present reference palette 
embedding invention are recovered by processing the digital image in machine 
readable, digital form. Human readable versions of images containing auxihary 
data, for example images displayed on a screen or printed from the digital data, 

15 cannot be processed to recover the embedded information. In a preferred 
embodiment of the subject invention, the auxiliary data are compressed and 
encrypted before beginning the reference palette embedding process, in order to 
randomize the auxihary bits, and to minimize the effect of the auxihary data on 
the difference between the Truecolor and color-palette images. 

20 It is therefore an object of the present invention to provide apparatus and 

method for embedding data into a digital information stream so that the 
meaning and content of the digital information stream is not changed 
significantly. 

It is another object of the present invention to provide apparatus and 
25 method for concealing auxihary data within a digital information stream so that 
the presence of the auxihary data is not discernible in the digital information 
stream. 
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It is yet another object of the present invention to provide apparatus and 
method for reducing the error caused by the added information, and for 
thwarting unauthorized access to the auxiliary data embedded into digital 
information stream. 

5 It is still another object of the present invention to provide apparatus and 

method for allowing authorized construction of embedded auxiliary data from a 
digital information stream- 
Additional objects, advantages, and novel features of the invention will be set 
forth in part in the description which follows, and in part will become apparent to 
10 those skilled in the art upon examination of the following, or learned by practice of 
the invention. The objects and advantages of the following, or learned by practice of 
the invention. 

The objects and advantages of the invention may be realized and attained by 
means of the instrumentalities and combinations particularly pointed out in the 
15 appended claims. 

SUMMARY OF THE INVENTION 
In accordance with the purposes of the present invention there is provided 
a method of embedding auxiliary data into pubhcation quahty digital image data 
represented by a quantity of color-component values for each picture element 

20 comprising the steps of reducing the quantity of color-component values of the 
publication quality digital image data to create a digital reference palette, 
wherein the digital color palette represents the quantity of color-component 
values of the pubhcation quality digital image data; creating a digital 
representation of the auxihary data as a sequence of individual bit values; 

25 creating a color-component digital difference image by numerically combining 
the publication quaUty digital image with the digital reference palette image; 
modifying the color-component digital difference image by combining the 
auxiliary data and the color-component digital difference image through use of a 
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data embedding method; creating a modified publication quality digital image 
indiscernibly containing the auxiliary data by combining the modified color- 
component digital difference image and the digital reference palette image; and 
outputting the modified pubHcation quahty digital image into a file format 
5 specified for the modified publication quality digital image. 

In a still further aspect of the present invention, and in accordance with 
its objects and purposes, a method of constructing indiscernible auxihary data 
firom a machine readable pubhcation quality digital image representation of 
unrelated and uncorrelated data comprising the steps of generating a digital 

10 reference palette image from values and properties contained within the 

pubhcation quahty digital image; creating a color-component digital difference 
image by numerically combining the digital reference palette image and the 
pubhcation quahty digital image; constructing the auxiliary data by processing 
the color-component digital difference image with a data embedding construction 

1 5 method; interpreting the auxihary data in order to obtain or remove content, 
vahdation or 

authentication, or otherwise process the pubhcation quahty digital image in 
order to modify its quahty. 

BRIEF DESCRIPTION OF THE DRAWINGS 
20 The accompanjdng drawings, which are incorporated in and form a part of 

the specification, illustrate the embodiments of the present invention and, 
together with the description, serve to explain the principles of the invention. In 
the drawings: 

FIGURE 1 is a diagram illustrating the reference palette sequence of 
25 calculations. 

FIGURE 2 is a partial hsting of computer code used for calculating the 
biased difference image color-component values. 

FIGURE 3 is a partial hsting of computer code used for calculating 
modified Truecolor image pixel color-component values. 
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FIGURE 4 is a diagram illustrating the sequence of calculation for constructing 
auxiliary data from a modified Truecolor image. 

FIGURE 5 is a partial listing of computer code used for constructing modified 
difference color-component values. 
5 DETAILED DESCRIPTION 

The present invention allows axixiliary data to be embedded into a digital 
Truecolor host image with less error than is caused by modifying the pixel color- 
components directly. The reduction in error follows from the technique of of the 
present invention of embedding auxihary data into the pixel color-component values 
1 0 constructed from the difference between the Truecolor host image and a reference 
palette image, which has been constructed from the Truecolor host image. The 
invention can be understood most easily through reference to the drawings. 

Refer to Figure 1 for an illustration of the process of the present invention. The 
images in Figure 1 are printed digital images, and are not copies of photographs. 
1 5 Pubhcation quahty digital image data 10, such as a Truecolor- format image is 

approximated or reduced by one of several commonly known color-reduction methods 

11 to produce a palette-format image 12. The palette-format image 12 is denoted 
hereinafter as reference palette image 12. The palette colors of reference palette image 

12 are subtracted from the Truecolor pixel color values of publication quality digital 
20 image 10 to create a difference image 13. The difference-image 13 pixel values 

measure directly the accuracy of the color-reduction method. Auxihary data 14 are 
taken as bits from a data source and input to data embedding processor JL5, which may 
contain the DATA EMBEDDING process, the MODULAR ERROR EMBEDDING 



wo 99/62044 



PCT/US99/09417 



process, or any other effective steganographic method for combining axixiliary data 14 
with difference image 13. A new difference image 13a is created by data embedding 
processor 15. The color values of the pixels in the new difference image 13a are added 
to reference palette image 12, and produce a new, modified publication quality digital 
5 image 10a, containing auxiliary data 14- 

Examples of appropriate publication quahty Truecolor format pubhcation 
quahty digital image data 10 include, but are not limited to, pubhcation quahty 
television or motion picture images. X-ray or Magnetic Resonance Imaging data, digital 
camera images, and personal security and identification data. Other examples of 

10 pubhcation quahty digital image data 10 include black and white images containing a 
range of digital levels of brightness, and digitized analog audio signals. For digitized 
audio signals, a reduced-quality version of the digitized analog audio signals serves as 
the reference palette 12. 

If the steganographic method used in data embedding processor 15 is bitshcing 

1 5 or the above-mentioned MODULAR ERROR EMBEDDING process, the first 

embodiment of the present invention is implemented. If the steganographic method 
used in DATA EMBEDDING processor 15 is the above-mentioned DATA 
EMBEDDING process, the second embodiment of the present invention is 
implemented. 

20 The difference image 13 is a Truecolor image, and negative pixel values are not 

permitted. Hence, the difference D between the Truecolor and pallet-color pixel colors 
is biased in the positive direction, in order to represent the difference as a positive 
number within the range 0-255 permitted for an 8-bit Truecolor-format image. The 
difference value is restricted to the range ±127, in order that the biased value remain 

25 within the 8-bit range. Pixels that are found to contain differences larger in absolute 
value than 127 are flagged, in order that the invention can place the original Truecolor 
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value in modified image 10a . Flagged pixels are not used by the invention, Figixre 2 is 
a partial listing of computer code in the C-H- language that is used for calculating the 
biased difference image color-component values. Figure 2 contains two nested loops 
starting at line 5, over the number of rows in the image, and at hne 13, over the 
5 number of columns in a row. 

The biased color differences are placed into a memory biaffer named buffer. 
Data from a row of Truecolor image pixels are placed into a memory buffer named 
bufftc in hne 7. The TCFile object is an instance of the MICROSOFT® MFC CFile:: 
class that accesses the bitmap-format Truecolor image. Data from the pictiure row in 

10 the palette-format image is read into a memory buffer named buffpal, from the 
CFile:: object named tape? at line 10, in Figure 2. 

The loop over the columns in the image row that begins at hne 13 in Figure 2 
processes the buffered pixel data. The three color-components in the Truecolor image 
pixel are processed sequentially within this loop. The index k contains the palette- 

15 format pixel value. The palette-format pixel colors are accessed by k, into the 

colormap[] array. The Truecolor pixel colors are accessed directly with offsets into 
the bufftc memory buffer. Color differences having a the value 255 are not used in 
data embedding processor 15. The biased color differences are b„diff, g„diff, and 
r„diff calculated at lines 17, 21, and 25. The differences are set to a limiting value 

20 (255) if the palette color values are greater than the arbitrary value of 250, i.e. the 

colors are near the top of their color ranges. The biased color differences are tested for 
range at lines 35, 41, and 47 in Figure 2. If the biased difference does not fit into the 
range 0-255 that is allowed by an 8-bit xmsigned character, the difference buffer is set 
to a flag value (0x01). Color differences that were set to the limiting value are flagged 

25 in this process. The color difference buffer becomes the output row in the Truecolor 
difference image. 
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Retxixning to Figure 1, the completed color difference image 13 i s combined with 
axixtliary data 14 by means of data embedding processor 15. In data embedding 
processor 15 this combination can be accomplished through use of bitshcing techniques, 
the above-mentioned MODULAR ERROR EMBEDDING process, the above-mentioned 
5 DATA EMBEDDING process, or any other effective steganographic algorithm. The 
color diflference image 13 is combined with reference palette image 12 to produce a new, 
modified Truecolor image 10a . 

Figure 3 is a partial listing of computer code used for calculating biased 
difference Truecolor-image-pixel color-component values. Two nested loops begin at 
1 0 line 5 and line 12 in Figure 3. The output buffer for the new, modified Truecolor image 
pixel row is named buffer. The difference image pixels are read into a memory buffer 
named bufftc at hne 7, from the CFile:: object named tape6. The palette-format pixel 
values are read into a btxffer named buffpal, from the CFile:: object named tape?, at 
hne 10. 

1 5 Construction of the new Truecolor image pixel row proceeds in the loop over 

image columns that starts at line 12. The output buffer is filled with the new color- 
value data. The statements contained in hnes 14 through 16 of Figure 3 process the 
first row of pixels differently, because the first image row is used to hold the key for the 
DATA EMBEDDING process. Processing the first row of pixels differently than the 

20 rest of the image is not part of the present reference palette embedding invention. 

The new Truecolor color-component values are calculated in hnes 19, 20, and 21, 
in Figure 3. Pixels in the difference image that contained flagged values are calculated 
incorrectly in this loop. The output buffer ofeets are set directly to the new color 
difference values, and the row of pixels is written to the new Truecolor image using the 

25 tapeS file object. 
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The tapes image file object is post-processed to replace the flagged pixels with 
the original Truecolor color-data pixel values. The flagged pixels, i.e. pixels that were 
not used to contain auxiliary data 14 (Figure 1), therefore appear without modification 
in the new Truecolor image 10a (Figure 1). 
5 Constructing (recovering) auxiliary data 14 fi:om new Truecolor image 10a 

requires the exact reference palette image 12 format version of original Truecolor 
image 10, and the information necessary to construct auxiliary data 14 fi^om new 
difference image 13a . Figure 4 is a diagram illustrating the sequence of calculation for 
constructing auxihary data 14 from a modified Truecolor image 10a. As in Figure 1, 

10 the images in Figm-e 4 are printed digital images, and are not copies of photographs. 

The coding to construct auxihary data 14, according to the process illustrated in 
Figtxre 4, is shown in Figure 5, The color difference image 13a is calculated from 
modified Truecolor inaage 10a and the reference palette image 12. The digital key is 
used with the data construction processor 15a to construct auxihary data 14. The 

15 method named MakeDifferenceFileO is executed at line 1 in Figure 5. The 

MakeDifferenceFileO method implements the calculation shown in Figure 2. The 
OpenBitmapFileO method executed at line 3 prepares the difference image 13a for 
processing by either the above-mentioned MODULAR ERROR EMBEDDING data 
construction process or the above-mentioned DATA EMBEDDING data construction 

20 process. The ExtractDataO method executed at line 4 in Figixre 5 constructs the 
auxihary data 14 fi:om the appropriate digital key and the difference image 13a. 

As with the DATA EMBEDDING process as taught in the above-mentioned US 
patent, another way of protecting the pair table key taught in that patent is to remove 
and encrypt it using pubhc-key or another encryption process. The present invention 
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requires the reference palette image 12, as well as the DATA EMBEDDING process 
key in order to construct auxiliary data 14- The necessary keys for DATA 
EMBEDDING process or codes for the steganography used to insert axixiUary data 14 
into the difference image can be combined with the reference palette image 12 using 
5 known and readily available file formats. The COMPUSERVE® Graphic Interchange 
Format™, the Tagged Image File Format, and the MICROSOFT® bitmap format 
enable the addition of additional binary information within the file header fields. 
Thus, the reference palette image 12 serves as the key to construct auxihary data 14 
from a pubhcation quality Truecolor version of the identical picture indiscernibly 

10 containing atixiliary data 14- 

The foregoing description of the embodiments of the invention have been 
presented for purposes of illustration and description. It is not intended to be 
exhaustive or to limit the invention to the precise form disclosed, and obviously many 
modifications and variations are possible in hght of the above teaching. The 

1 5 embodiments were chosen and described in order to best explain the principles of the 
invention and its practical appUcation to thereby enable others skilled in the art to best 
utiUze the invention in various embodiments and with various modifications as are 
suited to the particular use contemplated. It is intended that the scope of the invention 
be defined by the claims appended hereto. 
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WHAT IS CLAIMED IS: 

1. A method of embedding auxiliary data into publication quality digital image 
data represented by a quantity of color-component values for each picture element 
comprising the steps of: 

reducing said quantity of color-component values of said publication quality 
5 digital image data, to create a digital reference palette, wherein said digital color 

palette represents said quantity of color-component values of said publication quaUty 
digital image data; 

creating a digital representation of said auxiliary data as a sequence of 
individual bit values; 

10 creating a color-component digital difference image by numerically combining 

said publication quality digital image with said digital reference palette image; 

modifying said color-component digital difference image by combining said 
auxiliary data and said color-component digital difference image through use of a data 
embedding method; 

15 creating a modified publication quaUty digital image indiscernibly containing 

said auxiliary data by combining said modified color-component digital difference 
image and said digital reference palette image; and 

outputting said modified pubhcation quality digital image into a file format 
specified for said modified publication quality digital image. 

2. The method as described in Claim 1 further comprising the step of combining 
said auxihary data with predetermined information indicative of the presence of said 
auxiliary data, its file name, and file size, said step to be performed after the step of 
digitizing said auxihary data. 
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3. The method as described in Claim 1 further comprising the step of including an 
algorithm for removing or hiding a digital watermark signature into said modified 
publication quality digital image. 

4. The method as described in Claim 1, wherein said data embedding method 
comprises a bitsUce process. 

5. The method as described in Claim 1, wherein said data embedding method 
comprises a MODULAR ERROR EMBEDDING process. 

6. The method as described in Claim 1, wherein said data embedding method 
comprises a DATA EMBEDDING process. 

7. The method as described in Claim 1, wherein said publication quahty digital 
image originates from a pubhcation quality black and white image containing a range 
of digital levels of brightness. 

8 The method as described in Claim 1, wherein said publication quahty digital 
image originates from a digitized analog audio signal and said reference palette image 
originates from a reduced-quahty version of said digitized audio analog signal. 

9. The method as described in Claim 1, wherein said publication quahty digital 
image originates fr*om a television signal or motion picture image. 

10. The method as described in Claim 1, wherein said publication quahty digital 
image originates from X-ray or Magnetic Resonance Imaging data, 

11. The method as described in Claim 1, wherein said publication quahty digital 
image originates from digitized personal security and identification information. 

12. The method as described in Claim 1, wherein said pubhcation quahty digital 
image originates from images made with a camera producing digital images. 

13. A method of reconstructing indiscernible auxihary data from a machine readable 
publication quahtj'^ digital image representation of unrelated and uncorrelated data 
comprising the steps of: 
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generating a digital reference palette image from values and properties 
5 contained within said publication quality digital image; 

creating a color-component digital difference image by numerically combining 
said digital reference palette image and said publication quality digital image; 

constructing said auxiliary data by processing said color-component digital 
difference image with a data embedding construction method; 
10 interpreting said auxiliary data in order to obtain or remove content, vaUdate or 

authenticate, or otherwise process said publication quaUty digital image in order to 
modify auxiliary data quality. 
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// loop to calculate and store the biased-difference of the Truecolor 
image 

// difference = 128+(Truecolor - paiette) 
// create the Truecolor difference file 
5 pixelcount = OU 

for (i = 0; i < (short)bh.rows; i++) { 
memset(buffer, 0, BYTESJN_ROW); 

j = TCFIIe->Read(bufftc. BYTESJN_ROW);// Truecolor image row 
ASSERTO == (short)BYTESJN_ROW); 
10 bytesread += j; 

j = tape7.Read(buffiDal, bytesinrow); // palette-fonnat image row 
ASSERTQ == bytesinrow); 

for (j = 0; j < (short)bh.cols; { 
15 short b_diff. g_dlff. r_diff; 

char pixval[3]: 
k = *(buffpal+j): 

bjdiff = 128 + (short)*(bufftc + j * 3) - (short)colomiap[kl.b; 
if(colonnap[K|.b>250){ 
20 bjdiff = 255; //dont use maximum palette values 

} 

gjdiff = 128 + (short)*(bufftc + j * 3 + 1) - (short)colormap[k].g; 
lf(colomiap[k].g>250){ 
g_d!ff = 255; // dont use maximum palette values 

25 } 

r^diff = 128 + (short)*(bufftc + j * 3 + 2) - (short)coIomiap[kl.r; 

if (colonnap[kl.r > 250) { 
r_diff = 255; // don't use maximum palette values 

} 

FIGURE 2A 
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// set pixel to difference only if it is in range of unsigned char 
// otherwise flag with a value that is later removed from the pair- 

pixval[0] = pixval[1] = pixval[2] = \0'; 
if (b_diff < 255 && b_diff > 0) { 

*(buffer + j * 3) = (unsigned char)b_diff; 
} else { 

•(buffer + j * 3) = 0x01; // flag to mark out-of-range pixel 
pixval[0] = 'b'; 

} 

if (g_diff < 255 && g_diff > 0) { 

•(buffer + j * 3 + 1) = (unsigned char)g_diff; 
}else { 

•(buffer + j * 3 + 1) = 0x01: // flag to mark out-of-range pixel 
pixval[1] = 'g'; 

} 

If (r_diff < 255 && r_diff > 0) { 

•(buffer + j • 3 + 2) = (unsigned char)r_diff: 
} else { 

•(buffer + j • 3 + 2) = 0x01; // flag to mark out-of-range pixel 
plxval[2J = 'r'; 

} 
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// loop to calculate and store the output version of the Truecolor image 
// note: difference = 1 28+(Truecolor - palette) 
// hence: Truecolor = (difference -128) + palette 

for (i = 0; i < (short)bh.rows; i++) { 

memset(buffer. 0, BYTESJN_ROW); 

j = tape6.Read(bufflc, BYTES_IN_ROW); // difference image row 
ASSERT(j == (short)BYTESJN_ROW); 
bytesread += j; 

j = tape7.Read(buffi3al, bytesinrow); // palette-format image row 
ASSERT(j == bytesinrow); 
for (j = 0; j < (short)bh.cols; j++) { 
unsigned char b_diff, g_diff, r_diff; 
if(i==0){ 
memcpy(buffer,bufftc,BYTESJN_ROV\/); 
break; // difference embedding key in 1 st row 

} 

k = *(buffpal+j); 

b_diff = *(bufftc + j * 3) -128 + colormap^.b; 
g_diff = *(bufftc + j * 3 + 1) -128 + colormap[k].g; 
r_diff = *(bufftc + j * 3 + 2) -128 + colomiap[k].r: 
•(buffer + j * 3) = b_diff; 
*(buffer + j*3 + 1) = g_dlff; 
*(buffer + j*3 + 2) = r_diff; 

} 

tape8.Write(buffer, BYTESJN_ROW); // output one Truecolor 

image row 

} 
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MakeDifferencelmageO; 

// open the difference file and extract the pixel_table information 

ClmageBitmapFile::OpenBitmapFile(tempstr); 

ExtractBltmapO; 



FIGURE 5 
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